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The  Role  of  the  Coalition  Warrior  Interoperability 
Demonstration  in  the  Canadian  Forces  Joint  Experimentation 

Program 

by 

Sophie  Villeneuve,  Ronald  Funk  and  Kendall  Wheaton 


Abstract 

From  its  initial  charter  in  2001,  the  Canadian  Forces  Experimentation  Centre  (CFEC) 
inherited  the  responsibility  for  managing  the  Joint  Warrior  Interoperability 
Demonstration  (JWID)  program  within  Canada.  With  three  years  of  experience 
coordinating  JWID,  while  conducting  a  program  of  national  and  international 
experiments,  it  is  timely  to  review  the  role  of  this  event  in  CFEC’s  strategic  plan,  the 
Canadian  Forces  Joint  Concept  Development  and  Experimentation  Plan  (CF  JCD&E 
Plan).  This  paper  presents  a  strategy  for  integrating  the  JWID,  renamed  Coalition  Warrior 
Interoperability  Demonstration  (CWID)  in  2005,  into  the  CF  JCD&E  Plan  using  two 
trials  from  the  JWID  in  2004  as  case  studies.  These  trials  were  sponsored  by  the  Air 
Force  and  Defence  Intelligence  and  used  Defence  Research  and  Development  Canada 
(DRDC)  technologies.  The  Portal-to-Portal  Interoperability  Trial  used  a  situational 
awareness  portal  developed  by  the  COP  21  (Common  Operational  Picture  21st  Century) 
Technology  Demonstration.  The  Collaborative  Operations  Planning  System  Trial  tested  a 
prototype  for  providing  the  ability  to  plan  an  operation  in  a  net-enabled  environment 
using  integrated  collaborative  tools.  This  paper  concludes  with  discussion  of  a  strategy 
for  using  a  program  designed  to  trial  and  demonstrate  technologies  nearly  ready  for 
operational  use  in  a  Joint  Concept  Development  and  Experimentation  program  with  a 
mandate  to  support  transformation. 

Introduction 

The  Canadian  Forces  Experimentation  Centre  (CFEC)  is  responsible  for  the  development 
of  the  Canadian  Forces  Joint  Concept  Development  and  Experimentation  Plan  (CF 
JCD&E  Plan)  and  for  conducting  Joint  Concept  Development  and  Experimentation  in  the 
Canadian  Department  of  National  Defence  (DND).  From  its  initial  charter  in  2001, 
CFEC  inherited  responsibility  for  managing  the  Joint  Warrior  Interoperability 
Demonstration  (JWID)  program,  renamed  Coalition  Warrior  Interoperability 
Demonstration  (CWID)  in  2005.  This  parallels  the  situation  in  the  United  States  where 
the  U.S.  Joint  Forces  Command  (USJFCOM)  J9  Joint  Experimentation  took  over  the 
leadership  of  CWID  this  year.  With  three  years  of  experience  coordinating  JWID  while 
conducting  a  program  of  national  and  international  experiments,  it  is  timely  to  review  the 
role  of  JWID,  now  CWID,  in  the  CF  JCD&E  Plan. 

In  the  2004  JWID,  Canada  led  a  total  five  Coalition  Interoperability  Trials  (CITs), 
sponsored  either  by  the  Air  Force  or  by  Defence  Intelligence.  Some  of  the  technologies  in 
these  CITs  were  developed  by  Defence  Research  and  Development  Canada  (DRDC);  the 
agency  in  DND  for  Science  and  Technology  (S&T).  The  two  trials  selected  for 


discussion  in  this  paper  are  the  Portal-to-Portal  Interoperability  Trial,  sponsored  by 
Defence  Intelligence,  and  the  Collaborative  Operations  Planning  System  (COPlanS) 
Trial,  sponsored  by  the  Air  Force.  Both  used  DRDC  technologies  intended  to  transform 
Canadian  Forces  processes.  These  trials  were  selected  as  case  studies  since  both 
technologies  had  a  broader  role  in  Joint  Experimentation  in  DND. 

The  Canadian  technology  in  the  Portal-to-Portal  Interoperability  Trial  was  a  situational 
awareness  (SA)  portal  developed  by  the  Common  Operating  Picture  in  the  21st  Century 
Technology  Demonstration  (COP  21  TD).  This  application  was  developed  for  use  by  the 
Canadian  Forces  Joint  Staff  and  Joint  Operations  Group.  After  JWID  04,  CFEC  saw 
potential  for  the  COP  21  Portal  in  Joint  Forces  Command’s  Multinational 
Experimentation  (MNE)  Series  where  CFEC  had  taken  on  the  task  of  conducting  a 
Limited  Objective  Experiment  on  Knowledge  Management.  The  COP  21  Portal  has  since 
been  acquired  by  CFEC  for  further  experimentation. 

COPlanS  is  an  integrated  suite  of  planning,  decision-aid  and  workflow  management  tools 
designed  to  support  collaboration  by  distributed  military  staff  in  the  military  planning 
process.  An  early  prototype  had  been  tested  in  JWID  03.  Well  in  advance  of  JWID  04, 
this  application  had  received  support  from  CFEC  to  develop  this  tool  to  support  Effects 
Based  Planning  in  the  MNE  Series.  To  that  end,  CFEC  encouraged  and  assisted  in  the 
funding  for  the  COPlanS  CIT.  The  JWID  04  trial,  where  members  of  the  Canadian  Forces 
Joint  Staff  and  Joint  Operation  Group  would  be  the  primary  warfighter  audience,  was 
seen  as  ideal  for  preparation  for  the  next  MNE. 

The  formal  objective  of  future  CWIDs  is  to  demonstrate  technologies  that  should  be 
ready  for  operational  use  within  a  year.  Not  all  of  the  technologies  tested  will  be 
successful  and  so  some  will  take  longer.  Some  nations  participating  in  CWID  may  decide 
to  use  the  venue  for  promising  technologies  that  could  take  longer  anyway.  This  paper 
proposes  that  nations  should  consider  using  CWID  not  only  for  maturing  technologies 
ready  for  operations,  but  also  for  maturing  technologies  that  could  be  used  in 
experimentation.  While  both  of  the  technologies  in  these  case  studies  were  intended  for 
operational  use  in  the  near  term,  CFEC’s  experimentation  requirements  justified  a  test 
and  evaluation  trial  on  COPlanS  in  2004  and  JWID  provided  an  ideal  opportunity.  The 
COP  21  Portal  was  a  product  of  DRDC’s  Technology  Demonstration  Program  (TDP)  and 
JWID  04  provided  the  final  demonstration  venue  for  this  technology.  In  both  of  these 
cases,  the  JWID  venue  served  other  requirements  beyond  demonstrating  technologies  for 
operational  use. 

The  next  two  sections  will  describe  the  JWID  trials  for  Portal-to-Portal  Interoperability 
Trial  and  COPlanS.  The  JWID  assessment  process  was  applied  to  all  CITs.  Therefore,  a 
brief  explanation  of  this  process  is  required  for  background.  Three  types  of  assessments 
could  be  used  to  evaluate  each  JWID  trial.  The  Warfighter/Operator  Utility  Assessment 
(or  Warfighter  Assessment  as  it  is  normally  referred  to)  measured  the  technical 
performance  of  each  trial  in  terms  of  stated  objectives  and  capabilities  in  support  of  the 
overarching  JWID  objectives.  The  Interoperability  Assessment  evaluated  the  data 
exchanges  between  systems  to  ensure  that  the  data  transferred  was  received  and 
processed  correctly.  The  Security  Assessment  focused  on  how  the  trial  counters  threats 
and  enforces  policies  consistent  with  the  projected  environment.  In  the  discussions  to 
follow,  the  results  from  the  Warfighter  Assessments  will  be  presented.  The  data  for  this 


assessment  is  collected  during  JWID  execution  by  players  completing  network  accessible 
questionnaires.  These  questionnaires  are  generated  specifically  for  each  trial  based  on 
trial  objectives  (mapped  to  the  JWID  objectives),  trial  capabilities,  applicable  measures 
of  performance  (MOPs)  designed  for  each  trial,  and  predefined  scenario  events  and/or 
test  schedules. 

Trials  in  the  2004  Joint  Warrior  Interoperability  Demonstration 

(JWID) 

Portal-to-Portal  Interoperability  Trial  (CIT  04.05) 

Trial  Background  and  Objectives 

The  Common  Operating  Picture  in  the  21st  Century  is  a  Technology  Demonstrator  led  by 
DRDC  Valcartier,  aimed  at  improving  the  DND  Command  and  Control  (C2)  situational 
awareness.  More  specifically,  the  COP  21  TD  investigated  how  applying  a  systems 
architecture  approach  can  help  deliver  knowledge  management  concepts  and  advanced 
visualization  capabilities  using  portfolios  in  an  enterprise  level  portal  application.  This 
application  is  called  the  Situational  Awareness  (SA)  Portal. 

The  COP  2 1  TD  utilizes  a  range  of  technologies  that  can  only  be  adequately  assessed  by 
warfighters  in  a  complex  operational  setting.  JWID  04  provided  a  good  venue  for  the 
major  assessment  event  of  the  COP  21  TD.  CIT  04.05  (Portal-to-Portal  Interoperability) 
was  designed  to  test  as  many  aspects  as  possible  of  the  portals  and  visualization 
technologies.  This  paper  only  addresses  the  observations  made  on  the  COP  21  Portal, 
Canada’s  contribution  to  the  CIT. 

The  CIT  04.05  objectives  were  to  assess  the  portals’  capabilities  for  sharing  applications 
hence  information  leading  to  enhanced  situational  awareness  and  to  assess  the 
visualization  capabilities  of  six  visualization  applications.  This  involved  assessing: 

1.  The  portal’s  ability  to  help  the  users  to: 

a.  Discover,  gather,  filter,  and  view  the  appropriate  information;  and 

b.  Gain  situational  awareness; 

2.  The  interoperability  of  data  sources  and  services  with  other  portals  by: 

a.  Accessing  other  portals’  information; 

b.  Provision  of  COP  21  information  to  other  portals;  and 

c.  Integrating  web  services  from  other  organizations. 

This  trial  supported  the  JWID  Objective  on  ISR  dissemination  by  attempting  to  connect 
together  several  portals  in  a  distributed  fashion.  The  portals  involved  implemented 
different  approaches  to  provide  each  participating  organization  an  opportunity  to 
investigate  design  viewpoints  and  consider  the  interoperability  implications  these 
methodologies  might  pose. 


Data  Collection  and  Analysis  Plan 

The  analysis  plan  developed  for  CIT  04.05  used  three  data  collection  methods  to  support 
the  analysis: 

1.  Direct  observation  of  players  by  the  analysis  team; 

2.  Player  feedback  through  structured  surveys;  and 

3.  Quantitative  analysis  of  portal  data  captured  by  an  application  event  logger. 

The  Warfighter  Assessment  involved  the  analysis  team  monitoring  and  commenting  on 
each  event  of  the  scenario  event  list.  The  main  source  of  quantitative  data  came  from  a 
45  question  online  survey.  The  infonnation  was  collected  twice,  at  the  end  of  each  week 
of  execution.  The  CIT  04.05  assessment  questions  were  generic  so  as  to  apply  to  CIT 
participants  at  all  sites.  The  assessment  was  grouped  into  3  broad  areas;  portal  relevance 
to  regular  work,  to  JWID  work,  and  to  the  JWID  scenario. 

Portal  transaction  data  was  collected  in  an  event  logger  designed  specifically  for  the  COP 
2 1  Portal.  Key  transactions  were  parsed  into  fields  in  a  database  using  a  simple  analysis 
tool.  Metrics  and  associated  graphical  displays  were  developed  so  the  data  could  be 
converted  into  information  about  warfighter  and  system  performance.  The  process  and 
tools  successfully  demonstrated  that  portal  transactions  can  be  captured  quantitatively, 
measured  and  assessed.  The  analysis  of  the  transaction  data  is  still  in  progress  so  none  of 
the  following  results  relate  to  this  aspect  of  the  analysis  plan. 

Analysis  Results 

The  assessment  involved  35  warfighters  at  12  sites  in  6  countries.  The  Canadian  players 
were  experienced  operators  trained  in  the  Operational  Planning  Process  (OPP)  in  roles  in 
both  operational  and  strategic  level  headquarters.  The  players  outside  Canada  were  a 
good  cross-section  of  experienced  Canadian  and  other  coalition  nation  operators,  plus  a 
few  civilians/contractors  that  have  relevant  experience  working  with  the  military. 
Engineering  support  for  the  trial  was  available  at  the  main  site  in  Ottawa,  Canada  and  on¬ 
site  training  was  conducted  there  during  training  week.  All  warfighters  had  access  to  a 
comprehensive  on-line  help  file  within  the  portal  and  the  training  plan  called  for  the 
remainder  of  sites  to  be  briefed  using  an  online  collaboration  service. 

The  analysis  determined  that  CIT  04.05  successfully  supported  the  JWID  Objective  on 
ISR  Dissemination  as  well  as  the  JWID  Objective  on  Database  Fusion.  It  connected 
together  several  portals  and  shared  portal  applets  within  and  between  portals  in  a 
distributed  fashion.  The  portals  involved  used  various  implementation  approaches  to 
provide  each  participating  organization  an  opportunity  to  investigate  design  viewpoints 
and  consider  the  interoperability  implications  these  methodologies  might  pose.  Ottawa 
participants  felt  training  was  marginal  to  good  while  other  sites  were  critical  about  lack  of 
training  due  to  failures  of  the  collaboration  service  during  training  week. 

The  warfighters  believed  the  COP  21  Portal  improved  their  situational  awareness.  This 
was  accomplished  by  integrating  military  and  commercial  data  in  near  real  time  using 
various  tools  available  on  the  network.  Applications  involved  included  the  Coalition 
Portal  for  Imagery  and  Geospatial  Services  (CPIGS),  the  Request  For  Infonnation  (RFI) 
manager,  the  Geospatial  Information  System  (GIS)  browser,  and  the  COP  21 


visualization  tools  as  well  as  office  automation  tools  combined  in  a  single  desktop.  The 
portal  was  found  to  be  successful  at  using  existing  services  and  other  sources  plus 
exploiting  collaboration  tools.  The  portal  remained  stable  and  fully  available  during 
numerous  daily  scenario  events  throughout  the  JWID.  Players  easily  shared  information 
with  each  other  and  other  portals  at  several  sites. 

The  COP  2 1  Portal  employed  a  portfolio  concept  that  worked  very  well  at  integrating  and 
displaying  a  wide  variety  of  information  to  support  situational  awareness.  The  portfolios 
tended,  however,  to  become  disorganized  during  each  week  of  play.  It  was  decided  one 
player  should  have  been  assigned  the  role  of  content  manager.  The  search  tool, 
Autonomy,  demonstrated  the  capability  by  being  able  to  automatically  find  new  relevant 
documents  from  across  a  wide  range  of  unstructured  sources  using  contextual  searches. 
The  GIS  Browser  worked  properly  after  some  integration  issues  were  sorted  out  during 
the  first  week. 

The  following  six  visualization  applications  from  the  geospatial  community  were 
assessed: 

1.  Geoparsing  and  Retrieval  of  Indexed  Documents  (GRID)  to  search  for 
geospatial  locations  within  a  document; 

2.  ORB  AT  /  Infra  Browser  Display  for  information  of  military  and  non-military 
organizations  plus  information  about  critical  infrastructures; 

3.  GeoServNet  (GSN)  to  provide  3D  View  and  analysis  of  open  terrain  and 
urban  environments; 

4.  Strategic  Sight  to  provide  3D  Visualization  of  Strategic  level  information; 

5.  Tarentella  to  emulate  thick  applications  from  Defence  Threat  Reduction 
Agency  (DTRA);  and 

6.  Battle  Rhythm  to  allow  for  the  synchronization  of  critical  events  of  interest  to 
multiple  organizations. 

The  visualization  portion  of  the  trial  was  moderately  successful  in  demonstrating  the 
fusing  of  information  from  various  data  sources  into  a  homogenized  view  of  the  situation. 
Each  application  was  able  to  demonstrate  its  functionality  within  the  portal  but  several 
required  direct  support  from  developers  to  make  them  function  properly.  The  small 
number  of  scenario  events  and  corresponding  low  number  of  observations  led  to  the 
recommendation  that  the  visualization  capabilities  need  to  be  demonstrated  again  before 
conclusions  are  made  about  the  operational  utility  of  these  applications. 

The  players  gave  the  CIT  04.05  technologies  a  strong  endorsement  as  a  whole  and 
commented  very  positively  on  the  range  and  depth  of  functionality  that  was 
demonstrated.  They  also  provided  particularly  positive  feedback  about  the  flexibility 
and  utility  of  the  portfolio,  portal,  and  search  functions.  The  warfighters  made  many 
positive,  constructive  suggestions  about  how  to  improve  functionality,  particularly  about 
the  user  interface.  The  integration  of  applications  and  ease  of  access  to  them  was  also 
greatly  appreciated. 


Conclusions 

It  was  concluded  that  CIT  04.05  (Portal-to-Portal  Interoperability)  achieved  its  two 
planned  trial  objectives.  Sharing  portal  applications  between  portal  systems  was  a 
success.  The  trial’s  target  audience,  warfighters  supporting  operations  in  headquarters, 
felt  the  trial  demonstrated  capabilities  that  could  lead  to  major  improvements  in  their 
ability  to  maintain  situational  awareness  by  providing  “one-stop  shopping”  of  integrated 
information.  Although  it  took  time  for  the  players  to  become  comfortable  with  the  many 
capabilities  integrated  by  portfolios,  their  confidence  in  the  portal  services  increased 
quickly  to  the  point  where  they  soon  considered  it  to  be  the  preferred  way  to  do  business. 
The  improved  availability  associated  with  on-site  engineering  support  confirms  the  need 
to  ensure  adequate  engineering  support  when  fielding  any  sophisticated  technology  such 
as  a  portal.  The  demonstration  of  advanced  visualization  capabilities  was  more  limited 
but  was  still  considered  successful  given  its  stage  of  development. 

Recommendations 

The  assessment  recommended  that  the  COP  21  Portal  be  fielded  after  modifications  to 
take  into  account  the  new  requirements  proposed  by  the  JWID  2004  players.  The  existing 
version  of  the  portal  and  its  services  were  deemed  suitable  as  a  core  service  for  future 
experiments  and  demonstrations.  It  was  recommended  that  the  advanced  visualization 
capabilities  should  be  developed  further  and  tested  in  future  trials.  The  COP  21  Portal 
developers,  the  COP  21  TD,  were  encouraged  to  pursue  a  strategy  that  would  field  the 
existing  portal  as  quickly  as  possible. 


Collaborative  Operations  Planning  System  (COPIanS)  (CIT  04.08) 

Trial  Background  and  Objectives 

The  COPIanS  Trial  (CIT  04.08),  which  also  supported  the  JWID  objective  on  ISR 
Dissemination,  was  based  on  a  technology  designed  to  provide  the  ability  to  plan  an 
operation  in  a  net-enabled  environment  using  integrated  collaborative  tools.  It  is  a 
flexible  suite  of  planning,  decision-aid,  and  workflow  management  tools  capable  of 
supporting  a  distributed  team  in  the  conduct  of  mission  planning.  It  offers  functions  to 
design,  manage  and  synchronize  multiple  concurrent  battle  rhythms  at  the  strategic  and 
operational  levels,  and  to  a  limited  extent  at  the  tactical  level.  It  can  help  synchronize  the 
staff  workflow,  automatically  document  the  decision-making  process,  and  replay  the 
decision-path.  The  planning  tools  help  warfighters  to  sketch  courses  of  action  (COAs)  on 
maps,  to  perform  time  and  space  synchronization,  to  manage  resources  and  capabilities, 
to  manage  the  order  of  battle  (ORB AT)  and  to  perfonn  logistics  analyses.  The  decision- 
aid  tools  help  the  staff  to  rationalize  the  planning  process,  to  evaluate  and  compare  the 
COAs,  and  to  rapidly  produce  documents  to  support  the  Commander’s  decisions. 
COPIanS  has  collaborative  tools  including  chat,  white  board  and  a  map  planner.  A 
context  sensitive  search  engine  is  also  available  to  browse  past  similar  operations  and 
recall  plans  and  lessons  learned  from  the  database. 

The  objectives  of  the  COPIanS  trial  were  [Ref  1]: 


1.  To  demonstrate  and  evaluate  COPlanS  as  a  multi-level  workflow  management 
system,  a  distributed  OPP  collaborative  planning  system,  a  decision  support 
system,  and  as  an  information  broadcasting  system;  and 

2.  Within  the  JWID  environment: 

i.  to  support  distributed  planning  by  multiple  headquarters  engaged  in 
multiple  operations  at  the  operational  level; 

ii.  to  allow  coalition  HQs  to  collaborate  from  different  sites;  and 

iii.  to  allow  reach  back  to  different  sites  and  support  automatic 
dissemination  of  properly  formatted  information  (i.e.  Operation  Orders, 
Operation  Plans). 

The  trial  was  a  small  experiment  to  test  the  following  detailed  hypothesis  [Ref  1]: 

If  COPlanS  is  employed  by  multiple  HQs  in  a  net-enabled  operations  environment, 
then  geographically  distributed  staff  officers  will  be  able  to: 

•  Maintain  staff  synchronisation  while  executing  the  different  OPP  activities 
and  maintain  near-real  time  staff  updating  on  new  information  thereby 
improving  situation  awareness  as  well  as  the  tempo  and  quality  of  the 
decision-making; 

•  Perform  distributed  mission  analysis  in  a  structured  and  rational  manner 
thereby  improving  the  qualitv  of  the  staff  assessment  and  the  information  brief 
(when  combined  with  the  COP  21  Portal); 

•  Perform  distributed  full  scale  COA  development,  analysis  and  evaluation 
thereby  supporting  distributed  collaborative  planning  and  manual  war 
gaming  to  improve  the  COA  quality; 

•  Perform  distributed  COAs  comparison  and  produce  an  online  decision¬ 
briefing  thereby  rationalizing  the  decision-making  process  and  improving  the 
decision  quality;  and 

•  Replay  the  decision  path  and  thereby  support  organisation  memory 
(automatically  document  processes  and  decisions). 

Data  Collection  and  Analysis  Plan 

The  analysis  plan  for  CIT  04.08  was  based  on  the  first  two  of  the  three  collection 
methods  employed  for  CIT  04.05;  direct  observation  of  players  by  the  analysis  team  and 
player  feedback  through  structured  surveys.  The  data  collection  was  designed  based  upon 
metrics  identified  from  the  experiment  hypothesis  and  the  capabilities  of  the  technologies 
under  trial.  The  trial  attempted  to  quantify  these  metrics,  however,  data  collection  in  this 
experiment  was  limited  and  some  estimates  and  approximations  were  employed. 

Questionnaires  were  used  to  determine  the  level  of  staff  synchronisation  during  the 
execution  of  different  planning  activities.  The  latency  time  on  new  information  (as 
perceived  by  the  staff  officer)  was  to  be  estimated  by  observation.  Observers  and 
questionnaires  were  used  to  determine  the  level  of  situational  awareness  of  the 
participants  and  questionnaires  were  used  to  detemine  if  COPlanS  had  a  positive  impact 
on  the  tempo  and  quality  of  the  decision  making. 

Both  questionnaires  and  observations  were  used  in  order  to  characterize  the  distributed 
mission  analysis  activity  process  in  terms  of  structure  and  rational  and  to  assess  the 
quality  of  the  staff  assessment  and  quality  of  the  information  brief.  Players  were  also 


questioned  on  the  requirements  for  distributed  full  scale  course  of  action  (COA)  analysis, 
the  effectiveness  of  COPlanS  for  distributed  collaborative  planning  and  COA 
comparison,  the  quality  of  the  COA  produced  in  the  trial,  and  the  production  of  the  online 
decision  briefings  using  COPlanS.  Observations  and  questionnaires  were  both  used  to 
estimate  the  improvement  to  the  decision-making  process  and  decision  quality  in  this 
trial. 

As  JWID  is  a  warfighter  test,  evaluation  and  experimentation  venue  for  technology 
validation,  the  data  collection  is  designed  primarily  for  the  collection  of  warfighter 
feedback.  Surveys  typically  employ  a  series  of  questions  about  each  technology  function 
or  capability,  which  were  linked  to  the  metrics  for  CIT  04.08.  The  evaluation  of  the 
functions  or  capabilities  is  based  upon  the  warfighter  reactions  stimulated  by  the  events 
in  the  scenario  while  using  the  technology.  Consequently,  seven-point  agree  (also  known 
as  Lickert)  scales  were  used  extensively  in  the  surveys.  The  following  questionnaire  was 
developed  to  collect  warfighter  feedback  on  the  COPlanS  functions  described  in  the 
hypothesis  and  objectives. 

General: 

1.  In  general,  distributed  collaboration  is  useful  for  conducting  the  operational 
planning  process  (OPP)  collectively.  (Agree  Scale) 

2.  In  general,  the  operational  planning  process  (OPP)  provided  in  COPlanS 
corresponds  to  my  understanding  of  that  process.  (Agree  Scale) 

Synchronization  and  Situational  Awareness: 

3.  Situation  awareness  within  our  group  was  improved  using  COPlanS  compared 
to  existing  tools.  (Agree  Scale) 

4.  The  synchronization  of  the  staff  was  improved  using  COPlanS  to  execute  OPP 
activities  compared  to  existing  tools.  (Agree  Scale) 

5.  COPlanS  improved  the  tempo  of  the  decision  making  within  my  group. 
(Agree  Scale) 

6.  COPlanS  improved  the  quality  of  the  decision  making  within  my  group. 
(Agree  Scale) 

Distributed  Mission  Analysis: 

7.  COPlanS  supported  the  distributed  mission  analysis  activity  effectively. 
(Agree  Scale) 

8.  COPlanS  helped  to  improve  the  quality  of  the  staff  assessment  and  the 
information  brief.  (Agree  Scale) 

Full  Scale  COA  Development: 

9.  Is  COPlanS  a  suitable  tool  to  perform  distributed  full  scale  COA  development, 
analysis  and  evaluation?  (Yes/No)  If  not  please  explain. 

10.  COPlanS  allowed  me  to  collaborate  effectively  in  planning  activities  with  the 
national  and  coalition  partners  at  other  sites.  (Agree  Scale) 

11.  Did  COPlanS  improve  the  quality  of  the  COAs?  (Yes/No) 

COA  Comparison,  Brief  and  Decision: 

12.  COPlanS  was  effective  in  supporting  COAs  comparison  in  a  distributed 
environment.  (Agree  Scale) 


13.  COPlanS  was  effective  in  supporting  the  production  of  an  online  decision 
briefing.  (Agree  Scale) 

14.  COPlanS  helps  structure  and  improve  the  process  for  deciding  on  a  COA. 
(Agree  Scale) 

There  are  practical  limits  to  the  questions  that  can  be  addressed  through  survey 
questionnaires  and  to  the  level  of  effort  requested  from  the  players.  Observers  were 
therefore  tasked  to  comment  on  several  questions  plus  a  number  of  key  issues.  These  are 
included  in  the  full  data  collection  plan  for  the  trial  which  is  described  at  Ref  2. 

Analysis  Results 

COPlanS  was  employed  primarily  in  conjunction  with  two  other  CITs:  CIT  02.18 
Canadian  Air  Tasking  Order  /  Airspace  Control  Order  (ATO/ACO)  XML  Interpreter 
(CAAT-Xi)  and  CIT  04.05  (Portal-to-Portal  Interoperability).  (CIT  02.18  was  an 
application  designed  to  provide  operators  with  the  capability  to  display  US  or  NATO 
generated  ATO  and  ACO  data  in  a  graphical  and  tabular  fonnat.)  COPlanS  and  the  COP 
2 1  Portal  were  used  to  support  warfighters  playing  roles  in  a  Joint  Headquarters  and  roles 
as  Air  Force  mission  planners.  The  assessment  involved  19  players  at  7  sites  in  5 
countries.  The  majority  of  the  Joint  players  were  operators  assigned  to  Joint  units.  The 
Air  Force  players  were  a  mixture  of  operators  and  staff  employed  in  technical  support 
positions.  The  group  was  evenly  mixed  between  Joint  and  Air  Force,  but  largely 
Canadian.  Most  of  the  players  were  very  experienced  military  staff  familiar  with 
operations  or  operational  requirements. 

As  a  tool  that  supports  a  distributed  mission  planning  team  through  distributed 
collaborative  analysis  and  planning,  within  and  across  coalition  and  inter-agency 
information  domains,  COPlanS  was  found  to  be  effective  and  improve  the  quality  of  the 
staff  assessment  and  the  information  brief.  In  this  trial,  COPlanS  primarily  enhanced  the 
sharing  of  ISR  products  as  the  scenario  was  not  well  suited  for  testing  the  dissemination 
of  planning  products.  In  terms  of  improving  team  synchronization  and  situational 
awareness,  the  assessment  findings  indicated  that  COPlanS  improved  situational 
awareness  compared  to  existing  tools  as  well  as  synchronization  in  the  planning  process. 
While  COPlanS  was  not  seen  to  improve  the  tempo  of  the  planning  process  in  this  trial,  it 
was  judged  to  provide  some  improvement  in  the  quality  of  decisions. 

As  a  tool  for  COA  development,  the  players  felt  that  COPlanS  did  not  improve  COA 
quality  in  this  trial  and  that  it  had  little  impact  on  their  ability  to  collaborate  effectively  in 
COA  planning  with  staff  at  other  sites.  This  may  have  been  a  consequence  of  having 
insufficient  time  for  mission  planning  within  the  scenario  events  schedule.  They  did  feel, 
however,  that  it  was  a  suitable  tool  for  COA  development,  analysis  and  evaluation  in  a 
distributed  environment.  As  a  tool  that  supports  COA  selection  (the  decision  making 
process),  the  players  indicated  clearly  that  COPlanS  helped  structure  and  improve  the 
process  of  selecting  a  COA.  They  also  felt  that  COPlanS  was  useful  for  comparing 
COAs  in  a  distributed  environment  and  effective  in  the  online  production  of  a  decision 
brief. 

The  assessment  showed  that  warfighters  had  a  wide  range  of  reactions  to  this  CIT.  Some 
felt  the  tool  was  too  complex  for  their  needs  while  others  were  quite  enthusiastic  at  the 
potential  capability.  There  was  dissatisfaction  with  the  training,  the  scenario,  and  the  time 


available  to  conduct  planning  activities  with  the  tool.  Some  of  the  Joint  players  provided 
specific  comments  about  the  suitability  of  the  interface  and  ways  to  make  the  tool  more 
usable.  The  assessment  results  showed  that  on  average  the  players  approved  of  this  new 
technology,  but  wanted  to  see  more  development  before  it  is  fielded. 

Conclusions 

The  COPlanS  technology  is  still  developing  and  JWID  04  provided  a  valuable  trial 
opportunity  in  which  to  gather  warfighter  feedback  on  the  capabilities  in  a  distributed 
environment.  The  overall  feedback  was  positive  indicating  that  this  technology 
performed  well  in  the  trial  and  therefore  should  be  useful  to  the  warfighter.  Considerable 
constructive  criticism  was  collected  and  it  is  clear  that  there  should  be  more  development 
to  prepare  this  tool  for  operational  use.  Comments  were  received  that  COPlanS  should  be 
trialed  in  a  different  scenario  with  more  time  available  for  mission  planning.  It  was  also 
clear  that  players  require  significant  training  in  preparation  for  trials  using  such  a 
complex  tool. 

Recommendations 

COPlanS  should  be  considered  for  operational  employment  after  some  further 
development.  The  development  should  include  improvements  to  the  user  interface  and  to 
the  collaboration  services.  Plans  should  be  made  for  future  trials  that  employ  a  more 
suitable  scenario  for  mission  planning  and  provide  more  time  to  conduct  planning  and  the 
opportunity  to  test  effectiveness  of  the  tool  in  the  dissemination  of  the  products. 


Strategy  for  Integration  into  the  CF  JCD&E  Plan 

The  preceding  case  studies  will  now  be  considered  in  terms  of  the  strategic  plan  for  Joint 
Experimentation  in  the  Canadian  Forces.  The  Canadian  Forces  Experimentation  Centre 
was  established  with  the  specific  mission  to  lead  the  exploration  and  evaluation  of 
emerging  concepts  to  determine  the  capabilities  required  by  the  DND/CF  in  the  future. 
CFEC  sponsors  or  champions  activities  (concept  development,  experimentation,  and 
modeling  and  simulation)  that  relate  directly  to  the  accomplishment  of  this  mission.  With 
this  in  mind,  CFEC  has  the  following  mission  statement: 

As  the  Centre  of  Excellence  for  Joint  CD&E,  CFEC  will  lead  the  exploration  of 
emerging  concepts  and  the  experimentation  of  capabilities  that  support  CF 
transformation. 

Plan  Pegasus  04  is  the  CF’s  strategic  plan  that  details  intended  Joint  Concept 
Development  &  Experimentation  activity  for  the  coming  year  and  outlines  intentions  for 
the  next  several  years.  It  outlines  the  major  concepts  under  development  at  CFEC  in  the 
areas  of  Command  and  Control  (C2),  Information  and  Intelligence  (12),  and  Sustainment. 
Within  the  C2  program,  the  concepts  relate  to  interoperability  issues  associated  with 
operations  within  a  framework  that  is  joint,  inter-agency,  multinational  and  public. 
Interoperability  across  these  categories  should  occur  in  three  domains  [Ref  3]: 

•  Information  interoperability,  the  way  we  share  information  including 
technological  and  procedural  aspects; 


•  Cognitive  interoperability,  the  way  we  perceive  and  think  reflected  in  doctrine 
and  decision  processes;  and 

•  Behavioural  interoperability,  the  way  we  carry  out  the  selected  course  of 
action. 

CFEC’s  C2  experimentation  objectives  are  to  investigate  Command  and  Control 
interoperability  issues  at  the  operational  level  and  both  internally  and  with  external 
security  partners.  This  will  lead  to  recommendations  for  concept  implementation 
strategies  to  achieve  effective  and  efficient  employment  of  military  capabilities  through 
controlled  scientific  investigation.  As  stated  in  Plan  Pegasus  04  [Ref  3]: 

“If  the  CF  is  to  remain  interoperable  with  the  transforming  organizations  within 
the  US  and  other  close  allies,  then  we  must  understand  fundamental 
transformational  concepts  and  implementation  strategies  within  their 
organizations  and  their  associated  interoperability  implications,  and 
appropriately  modify  selected  concepts  and  their  implementation  strategy  for 
Canadian  adoption.  ” 

Within  the  timeframe  of  Plan  Pegasus  04,  the  CFEC’s  C2  experimentation  campaign 
plan  will  focus  on  national  involvement  in  multinational  events  such  as  the  MNE  series, 
JWID  04,  and  the  CWID  series  starting  in  2005.  The  JWID/CWID  series  focus  on  near 
term  (one  to  five  years)  concepts  and  implementation  strategies  associated  with 
information  sharing  and  collaboration  while  the  MNE  series  investigates  near  to  mid  term 
(five  to  ten  years)  concepts  and  implementation  strategies  for  Effects  Based  Approaches 
to  Security  and  Defence  within  a  multinational  coalition.  These  activities  are  seen  to 
facilitate  the  investigation  of  interoperability  issues  associated  with  the  requirement  for 
military  operations  conducted  within  a  framework  that  is  joint,  inter-agency, 
multinational  and  public. 

The  challenge  is  to  provide  validated  concepts  through  integrated  analysis  and 
experimentation  exploiting  modeling  and  simulation  as  much  as  possible.  The  United 
States  Joint  Forces  Command  J9,  Joint  Experimentation,  uses  two  pathways:  a  Concept 
Development  pathway  and  a  Prototype  pathway  [Ref  4].  The  concept  pathway  uses  a 
series  of  events  with  the  overarching  objective  of  developing  a  shared  vision  through 
collaboration  with  the  combatant  commanders,  the  services  and  USJFCOM.  USJFCOM 
refers  to  this  as  the  “common  joint  context”.  The  common  joint  context  is  used  to  create  a 
common  joint  experimentation  environment  through  integrating  events  that  take  the 
knowledge  acquired  from  service  games  (exploring  the  common  joint  context)  into  a  joint 
environment.  The  service  events  shape  the  capabilities  while  the  joint  events  integrate  the 
capabilities. 

It  is  very  important,  from  USJFCOM ’s  point  of  view,  to  bring  as  many  experts  as 
possible  into  the  concept  development  stage  to  ensure  truly  joint  functionality  across 
command  and  services.  The  best  ideas  from  the  concept  pathway  are  used  on  the 
prototype  pathway  where  products  and  capabilities  are  developed  to  support  validated 
concepts.  Products  on  the  prototype  pathway  are  developed  and  tested  through  an 
experimentation  process  with  input  from  senior  officials  from  the  various  armed  services, 
USJFCOM  and  defence  leaders. 


Like  USJFCOM,  CFEC  needs  both  concept  development  and  prototyping  venues  to 
evolve  the  products  associated  with  concept  development.  It  is  acknowledged  within  the 
MNE  community,  that  although  the  events  in  this  series  are  part  of  J9’s  prototype 
pathway,  they  are  generally  concept  development  opportunities  for  the  other  nations.  It  is 
therefore  proposed  that  CWID,  an  environment  traditionally  for  prototypes  about  to  be 
taken  into  operational  use,  could  also  be  suitable  for  prototype  products  from  concept 
development  environments. 


Conclusion 

The  case  studies  presented  in  this  paper  describe  the  successful  trials  of  two  technologies 
that  have  potential  for  concept  development  experiments.  The  COP  2 1  Portal  was  tested 
in  JWID  04  as  part  of  the  evolution  of  DRDC’s  Technology  Demonstration  Program.  It 
was  very  successful  in  that  demonstration  and  provided  a  capability  for  net-centric 
operations  for  an  operational  level  headquarters.  CFEC  subsequently  considered  using 
the  COP  21  Portal  in  their  Knowledge  Management  experimentation  for  MNE  4. 
COPlanS,  a  tool  that  supports  the  mission  planning  process,  was  proposed  by  Canada  in 
June  2003  for  use  in  MNE  3  with  consideration  for  employment  in  MNE  4  as  well.  The 
reconfigurable  features  of  this  application  allow  it  to  be  applied  to  any  planning  process: 
traditional  operational  planning  or  the  planning  for  missions  in  Effects  Based  Operations. 

JWID  04  provided  the  ideal  testing  environment  for  tools  that  could  be  used  in  a  joint 
operational  headquarters  relying  on  distributed  collaboration  to  conduct  mission  planning 
or  operations.  Subsequent  to  a  successful  concept  development  experiment,  tools  should 
be  trialed  in  CWID  again  for  validation  prior  to  integration  into  operational  systems.  CIT 
04.05  showed  that  the  collection  of  transaction  data  for  subsequent  detailed  analysis  has 
significant  potential;  this  should  be  developed  further. 

CWID  provides  opportunities  for  exposing  prototypes  to  the  operational  community, 
however,  there  are  limitations.  No  single  event  can  meet  all  the  requirements  of  the 
CD&E  community.  Likewise,  both  pathways  are  needed  to  produce  well-rounded 
concepts  and  tools.  CWID  can  support  experiments  like  MNE4  and  in  turn  applications 
that  show  well  in  events  like  MNE4  can  be  re-trialed  in  CWID  before  being  put  into 
operational  use.  With  careful  planning,  these  venues  can  be  used  iteratively  to  progress 
CD&E  in  the  joint  environment. 
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Canadian  Forces  Experimentation  Centre 


•  CFEC  is  responsible  for  the  development  of  the 
Canadian  Forces  Joint  Concept  Development  and 
Experimentation  Plan  (CF  JCD&E  Plan)  and  for 
conducting  Joint  Concept  Development  and 
Experimentation  in  the  Canadian  Department  of 
National  Defence  (DND). 

•  From  its  initial  charter  in  2001,  CFEC  inherited 
responsibility  for  managing  the  Joint  Warrior 
Interoperability  Demonstration  (JWID)  program, 
renamed  Coalition  Warrior  Interoperability 
Demonstration  (CWID)  in  2005. 

•  This  parallels  the  situation  in  the  United  States 
where  the  U.S.  Joint  Forces  Command 
(USJFCOM)  J9  Joint  Experimentation  took  over 
the  leadership  of  CWID  this  year. 
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JWID 


What  is  JWID? 


JWID  is  the  US  Chairman  of  the  Joint  Chiefs  of 
Staffs  annual  event  that  enables  the  US  combatant 
commands  and  the  international  community  to 
investigate  command,  control,  communications, 
and  computer  (C4)  solutions  that  focus  on  relevant 
and  timely  objectives  for  enhancing  coalition 
interoperability. 

The  intent  is  to  conduct  trials  of  systems  that  can 
then  be  quickly  transitioned  into  operational  use. 
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JWID 


Case  Studies 
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•  Canada  led  a  total  of  five  Coalition  Interoperability 
Trials  (CITs),  sponsored  either  by  the  Air  Force  or 
by  Defence  Intelligence. 

•  The  two  trials  presented  in  this  paper  used 
technologies  developed  by  Defence  Research  and 
Development  Canada  (DRDC).  They  are: 


-  the  Portal-to-Portal  Interoperability  Trial;  and 

-  the  Collaborative  Operations  Planning  System 
(COPlanS)  Trial. 

These  trials  were  selected  as  case  studies  since  both 
technologies  had  a  broader  role  in  Joint 
Experimentation  in  DND. 
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JWID 


Case  Studies 
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The  Canadian  technology  in  the  Portal-to-Portal 
Interoperability  Trial  was  a  situational  awareness 
(SA)  portal  developed  by  the  Common  Operating 
Picture  in  the  21st  Century  Technology 
Demonstration  (COP  21  TD).  This  application  was 
developed  for  use  by  the  Canadian  Forces  Joint 
Staff  and  Joint  Operations  Group. 

COPlanS  is  an  integrated  suite  of  planning, 
decision-aid  and  workflow  management  tools 
designed  to  support  collaboration  by  distributed 
military  staff  in  the  military  planning  process.  An 
early  prototype  was  tested  in  JWID  03. 
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JWID 


Use  of  Venue 
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•  The  formal  objective  of  JWID/CWID  is  to 
demonstrate  technologies  that  should  be  ready  for 
operational  use  within  a  year. 

•  Not  all  of  the  technologies  tested  will  be  successful 
and  so  some  will  take  longer. 


This  paper  proposes  that  nations  should  consider 
using  CWID  not  only  for  maturing  technologies 
ready  for  operations,  but  also  for  maturing 
technologies  that  could  be  used  in  experimentation. 

In  both  of  these  case  studies,  the  JWID  venue 
served  other  requirements  beyond  demonstrating 
technologies  for  operational  use. 
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JWID  Assessment  Process 
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Three  types  of  assessments: 

-  Warfighter/Operator  Utility  Assessment 

-  Interoperability  Assessment 

-  Security  Assessment 

Warfighter  Assessment  questionnaires  are 
generated  specifically  for  each  trial  based  on: 

-  trial  objectives 

-  trial  capabilities 

-  applicable  measures  of  performance  (MOPs) 
designed  for  each  trial 

-  predefined  scenario  events  and/or  test  schedules 
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JWID  04  Trials 


•  Portal-to-Portal  Interoperability  Trial  (CIT  04.05) 

•  Collaborative  Operations  Planning  System  (CIT  04.08) 


Trial  Background  and  Objectives 
Data  Collection  and  Analysis  Plan 
Analysis  Results 
Conclusions 
Recommendations 
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Portal-to-Portal  Interoperability 

Trial  Background  and  Objectives 


•  The  COP  2 1  TD  investigated  how  applying  a 
systems  architecture  approach  could  help  deliver 
knowledge  management  concepts  and  advanced 
visualization  capabilities  using  portfolios  in  an 
enterprise  level  portal  application  -  known  as  the 
COP  21  Portal. 


Trial  objectives: 

-  Assess  the  portals’  capabilities  for  sharing 
applications 

-  Assess  the  visualization  capabilities  of  six 
visualization  applications 
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Portal-to-Portal  Interoperability 

Data  Collection  and  Analysis  Plan 


The  analysis  plan  developed  for  CIT  04.05  used  three 
data  collection  methods: 

1 .  Direct  observation  of  players  by  the  analysis 
team; 


2.  Player  feedback  through  structured  surveys;  and 


3. 


Quantitative  analysis  of  portal  data  captured  by 
an  application  event  logger. 
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Results  from  CIT  04.05 
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Question 

Week  1 

Avc  t  Std  Dev 

Week  2 

Ave  ±  Std  Dev 

Portal 

The  portal  capability  significantly  improved  my  situational 
awareness  in  JWID. 

5.37  ±  1.28 

5.71  i  1.46 

The  customization  features  allowed  me  to  significantly  improve  the 
utility  to  my  portal  work. 

4.61  ±  1.17 

4.42  ±  1.21 

The  portal  was  able  to  handle  production  activities  associated  with 
managing  information. 

4.80  ±  1.63 

5.68  ±  1.03 

My  access  to  the  production  tools  from  the  portal  was 
straightforward. 

4.54  ±  1.27 

5.21  ±  1.59 

Portfolio 

The  portfolio  approach  to  collating  information  improved  my 
situational  awareness  of  the  operations. 

5.89  ±  1.15 

5.92  ±  1.06 

The  portfolio  features  improved  my  ability  to  contnbute  to  JWID 
operations. 

5.80  ±  1.04 

5.74  ±  1.29 

Search  and  discovery 

The  search  tool  produces  sensible  summary  information  that  eases 
operator  workload. 

5.40  ±  1.19 

5.17±  1.15 

The  feature  that  uses  prior  search  results  to  extend  contextualized 
searches  contnbuted  to  the  discovery  of  now  information. 

5.18  ±  1.26 

5.17  ±  1.03 

The  query  agents  that  automatically  search  for  new  or  updated 
information  saved  me  significant  time  and/or  effort  to  discover 
information  needed  to  conduct  operations. 

5.52  ±  1.21 

5.00  ±  1.17 

The  five  icons  displayed  with  each  search  result  item  allowed  me  to 
use  different  perspectives  to  better  exploit  the  data. 

5.32  ±  1.31 

5.13±  1.14 

The  quality  of  search  results  contributed  to  my  situational 
awareness. 

5.39  ±  1.12 

5.22  ±  1.17 

Results  from  CIT  04.05 
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Question 

Week  1 

Avc  ±  Std  Dev 

Week  2 

Ave  t  Std  Dev 

GIS  Browser 

The  GIS  Browser  interface  with  the  C2PC  interface  helped  me  to 
improve  my  situational  awareness  by  allowing  me  to  display  unit 
locations  with  other  geo-spatial  information  in  the  portal. 

4.91  ±  1.27 

4.83  ±  1.19 

1  found  the  GIS  Browser  was  responsive  to  retneving  and 
displaying  new  data. 

4.92  ±  1.22 

4.83  ±  1.20 

The  GIS  Browser  allowed  me  to  select  the  appropnate  geospatial 
layers  in  order  for  me  to  build  a  customized  map. 

4.65  ±  1.11 

4.77  ±  1.02 

1  found  useful  the  possibility  to  set  bookmarks  of  geospatial  views. 

4.68  ±  1.04 

4.77  ±  1.23 

Integration  with  Other  Tools  in  the  Portal 

The  associated  stand-alone  applications  (i.e.  GRID,  RFI  Manager, 
Incident  Management  System  (IMS))  were  well  integrated  within  my 
portal 

4.92  ±  1.21 

4.83  ±  1.20 

General 

The  training  that  1  received  dunng  training  week  pnor  to  JWID 
execution  was  sufficient  to  complete  my  assigned  tasks. 

3.33  ±  1.80 

3.92  ±  1.75 

The  capabilities  of  the  portal  to  portal  trial  are  an  improvement  over 
current  capabilities  1  use. 

5.25  ±  1.29 

5.25  ±  1.36 

This  portal  to  portal  trial  should  be  taken  to  the  field  in  its  current 
configuration. 

4.61  ±  1.71 

4.96  ±  1.49 

The  portal  to  portal  process  worked  well  dunng  JWID. 

5.59  ±  1.22 

5.64  ±  1.44 

Overall,  the  portal  to  portal  CIT  provided  value-added  to  perform  my 
JWID  tasks.  List  the  specific  tasks  and  how  they  contnbuted. 

5.33  ±  1.14 

5.42  ±  1.50 
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Portal-to-Portal  Interoperability 

Analysis  Results 

CIT  04.05  successfully  supported  the  JWID  Objectives  on 
ISR  Dissemination  as  well  as  on  Database  Fusion. 

Connected  together  several  portals  and  shared  portal 
applets  within  and  between  portals  in  a  distributed  fashion. 

The  portals  involved  used  various  implementation 
approaches  to  provide  each  participating  organization  an 
opportunity  to  investigate  design  viewpoints  and  consider 
the  interoperability  implications  these  methodologies  might 
pose. 

Ottawa  participants  felt  training  was  marginal  to  good 
while  other  sites  were  critical  about  lack  of  training  due  to 
failures  of  the  collaboration  service  during  training  week. 
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Portal-to-Portal  Interoperability 

Analysis  Results 


•  COP  2 1  Portal  employed  a  portfolio  concept  that  worked  very 
well  at  integrating  and  displaying  a  wide  variety  of 
information  to  support  situational  awareness. 

•  Was  strongly  endorsed  with  positive  comments  on  range  and 
depth  of  functionality 


Positive  feedback  on  flexibility  and  utility  of  the  portfolio, 
portal  and  search  functions 

Integration  of  applications  and  ease  of  access  was 
appreciated  by  users. 

The  visualization  portion  was  moderately  successful  in 
demonstrating  the  fusing  of  information  from  various  data 
sources. 
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Portal-to-Portal  Interoperability 

Conclusions 


R 0 


Trial  achieved  its  two  planned  trial  objectives: 

-  Sharing  portal  applications  between  portal 
systems  was  a  success 

-  The  demonstration  of  advanced  visualization 
capabilities  was  more  limited  but  was  still 
considered  successful 

With  time,  players  became  comfortable  which 
quickly  increased  their  confidence  in  the  portal 
services. 


•  Need  adequate  engineering  support  when  fielding 
any  sophisticated  technology 
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Portal-to-Portal  Interoperability 

Recommendations 


•  COP  21  Portal  could  be  fielded  after  modifications. 


•  The  existing  version  of  the  portal  and  its  services 
were  deemed  suitable  as  a  core  service  for  future 
experiments  and  demonstrations. 


The  advanced  visualization  capabilities  should  be 
developed  further  and  tested  in  future  trials. 

The  COP  21  Portal  developers,  the  COP  21  TD, 
were  encouraged  to  pursue  a  strategy  that  would 
field  the  existing  portal  as  quickly  as  possible. 
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COPlanS 

Trial  Background  and  Objectives 


•  Also  supported  the  JWID  objective  on  ISR 
Dissemination 


•  Based  on  a  technology  designed  to  provide  the 
ability  to  plan  an  operation  in  a  net-enabled 
environment  using  integrated  collaborative  tools 


Flexible  suite  of  planning,  decision-aid,  and 
workflow  management  tools 

Can  help  synchronize  the  staff  workflow, 
automatically  document  the  decision-making 
process,  and  replay  the  decision-path. 


1/3 
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COPlanS 

Trial  Background  and  Objectives 


•  The  planning  tools  help  warfighters  to  sketch 
courses  of  action  (COAs).  The  decision-aid  tools 
help  the  staff  to  rationalize  the  planning  process,  to 
evaluate  and  compare  the  COAs,  and  rapidly 
produce  documents  to  support  the  Commander’s 
decisions. 


Has  collaborative  tools  including  chat,  white  board 
and  a  map  planner.  A  context  sensitive  search 
engine  is  also  available. 
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COPlanS 

Trial  Background  and  Objectives 
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1.  To  demonstrate  and  evaluate  COPlanS  as  a  multi-level 
workflow  management  system,  a  distributed  OPP 
collaborative  planning  system,  a  decision  support  system, 
and  as  an  information  broadcasting  system;  and 

2.  Within  the  JWID  environment: 

i.  to  support  distributed  planning  by  multiple  headquarters 
engaged  in  multiple  operations  at  the  operational  level; 

ii.  to  allow  coalition  HQs  to  collaborate  from  different 
sites;  and 

iii.  to  allow  reach  back  to  different  sites  and  support 
automatic  dissemination  of  properly  formatted 
information  (i.e.  Operation  Orders,  Operation  Plans). 
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COPlanS 

Data  Collection  and  Analysis  Plan 


•  The  analysis  plan  for  CIT  04.08  was  based  on 


-  direct  observation  of  players  by  the  analysis 
team 

-  player  feedback  through  structured  surveys 


The  data  collection  was  designed  based  upon 
metrics  identified  from  the  experiment  hypothesis 
and  the  capabilities  of  the  technologies  under  trial. 

The  trial  attempted  to  quantify  these  metrics, 
however,  data  collection  in  this  experiment  was 
limited  and  some  estimates  and  approximations 
were  employed. 
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COPlanS 

Data  Collection  and  Analysis  Plan 


•  Both  methods  were  used  to: 


-  determine  the  level  of  situational  awareness  of 
the  participants 

-  characterize  the  distributed  mission  analysis 
activity  process 


assess  the  quality  of  the  staff  assessment  and 
quality  of  the  information  brief 

estimate  the  improvement  to  the  decision¬ 
making  process  and  decision  quality  in  this  trial 
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Results  from  CIT  04.08 
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Question 

Week  1 

Avc  t  Std  Dev 

Week  2 
Avc  ±  Std  Dev 

Calibration 

In  general,  distnbuted  collaboration  is  useful  for  conducting  the 
operational  planning  process  (OPP)  collectively. 

5.47  ±  1.61 

5.47  ±  1.68 

In  general,  the  operational  planning  process  provided  in  COPIanS 
corresponds  to  my  understanding  of  that  process. 

6.00  ±  0.88 

5.79  ±  1.13 

Synchronization  and  Situational  Awareness 

Situation  awareness  within  our  group  was  improved  using  COPIanS 
compared  to  existing  tools. 

4.84  ±  1.57 

4.42  ±  1.26 

The  synchronization  of  the  staff  was  improved  using  COPIanS  to 
execute  OPP  activities  compared  to  existing  tools. 

4.84  ±  1.38 

4.63  ±  1.64 

COPIanS  improved  the  tempo  of  the  decision  making  within  my 
group. 

4.11  ±  1.20 

4.00  ±  1.49 

COPIanS  improved  the  quality  of  the  decision  making  within  my 
group. 

4.42  ±  1.35 

4.63  ±  1.46 

Distributed  Mission  Analysis 

COPIanS  supported  the  distnbuted  mission  analysis  activity 
effectively. 

4.84  ±  1.46 

4.63  ±  1.01 

COPIanS  helped  to  improve  the  quality  of  the  staff  assessment  and 
the  information  brief. 

4.79  ±0.92 

4.79  ±  1.08 

Results  from  CIT  04.08 
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Question 

Week  1 

Avc  ±  Std  Dev 

Week  2 
Avc  ±  Std  Dev 

Full  Scale  Course  Of  Action  (COA)  Development 

COPIanS  allowed  me  to  collaborate  effectively  in  planning  activities 
with  the  national  and  coalition  partners  at  other  sites. 

3.79  ±  1.69 

4.47  ±  1.65 

COA  Comparison,  Brief  and  Decision 

COPIanS  was  effective  in  supporting  COAs  comparison  in  a 
distnbuted  environment. 

4.63  ±  1.64 

4.84  ±0.96 

COPIanS  was  effective  in  supporting  the  production  of  an  online 
decision-bnefing. 

4.68  ±  1.73 

4.89  ±  1.70 

COPIanS  helps  structure  and  improve  the  process  for  deciding  on  a 
COA. 

5.21  ±1.69 

5.21  ±  1.36 

General 

The  training  that  1  received  dunng  training  week  pnor  to  JWID 
execution  was  sufficient  to  complete  my  assigned  tasks  with 
COPIanS. 

2.79  ±1.32 

3.32  ±  1.34 

The  on-line  help  provided  with  COPIanS  was  sufficient  to  complete 
my  assigned  tasks. 

4.21  ±  1.55 

3.79  ±  1.08 

The  scenario  and  MSELs  were  appropnate  for  the  COPIanS  trial 
plan  and  worked  well  dunng  JWID. 

3.68  ±  1.60 

3.74  ±1.19 

The  capabilities  provided  by  COPIanS  are  an  improvement  over 
current  capabilities  1  use.  Describe  the  capabilities  you  currently  use. 

4.74  ±  1.52 

4.74  ±  1.33 

COPIanS  should  be  taken  to  the  field  in  its  current  configuration. 

3.58  ±  1.61 

3.16  ±  1.61 

COPlanS 


Analysis  Results 


•  Found  to  be  effective  and  improve  the  quality  of 
the  staff  assessment  and  the  information  brief 


•  Primarily  enhanced  the  sharing  of  ISR  products  as 
the  scenario  was  not  well  suited  for  testing  the 
dissemination  of  planning  products 


Improved  situational  awareness  compared  to 
existing  tools  as  well  as  team  synchronization  in 
the  planning  process 

Although  not  seen  to  improve  the  tempo  of  the 
planning  process  in  this  trial,  it  was  judged  to 
provide  some  improvement  in  the  quality  of 
decisions. 
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COPlanS 


Conclusions 


•  The  COPlanS  technology  is  still  developing. 


•  The  overall  feedback  was  positive. 

•  More  development  is  required  to  prepare  this  tool 
for  operational  use. 


Comments  were  received  that  COPlanS  should  be 
trialed  in  a  different  scenario  with  more  time 
available  for  mission  planning. 

Players  require  significant  training  in  preparation 
for  trials  using  such  a  complex  tool. 
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COPlanS 


Recommendations 


COPlanS  should  be  considered  for  operational 
employment  after  some  further  development. 

The  development  should  include  improvements  to 
the  user  interface  and  to  the  collaboration  services. 

Plans  should  be  made  for  future  trials  that  employ  a 
more  suitable  scenario  for  mission  planning  and 
provide  more  time  to  conduct  planning  and  the 
opportunity  to  test  effectiveness  of  the  tool  in  the 
dissemination  of  the  products. 
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Outline 


•  Introduction 

•  Joint  Warrior  Interoperability 
Demonstration  2004  (JWID  04)  Trials 

•  Strategy  for  Integration  into  the  Canadian 
Forces  Joint  Concept  Development  and 
Experimentation  Plan 

•  Conclusion 
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Strategy  for  Integration  into  the  Canadian 
Forces  Joint  Concept  Development  and 
Experimentation  Plan 

•  CFEC’s  role 

•  Plan  Pegasus  04  Implications 

•  USJFCOM’s  Pathways 

•  Potential  Way  Ahead  for  CFEC 
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CFEC 
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RO 


CFEC  was  established  with  the  specific  mission  to 
lead  the  exploration  and  evaluation  of  emerging 
concepts  to  determine  the  capabilities  required  by 
the  DND/CF  in  the  future. 

CFEC  sponsors  or  champions  activities  that  relate 
directly  to  the  accomplishment  of  this  mission. 


Mission  Statement: 


As  the  Centre  of  Excellence  for  Joint  CD&E, 
CFEC  will  lead  the  exploration  of  emerging 
concepts  and  the  experimentation  of  capabilities 
that  support  CF  transformation. 
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r^y  Plan  Pegasus  04 


*  Plan  Pegasus  04  is  the  CF’s  strategic  plan  that 
details  intended  JCD&E  activity  for  the  coming 
year  and  outlines  intentions  for  the  next  several 
years. 

•  It  describes  the  major  concepts  under  development 
at  CFEC. 


CFEC’s  C2  experimentation  objectives  are  to 
investigate  C2  interoperability  issues  at  the 
operational  level  and  both  internally  and  with 
external  security  partners. 
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Plan  Pegasus  04 
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•  CFEC’s  C2  experimentation  campaign  plan  will 
focus  on  national  involvement  in  multinational 
events  such  as  the  MNE  series,  JWID  04,  and  the 
CWID  series  starting  in  2005. 

•  JWID/CWID  series  focus  on  near  term  (one  to  five 
years)  concepts  and  implementation  strategies 
associated  with  information  sharing  and 
collaboration. 

•  MNE  series  investigates  near  to  mid  term  (five  to 
ten  years)  concepts  and  implementation  strategies 
for  Effects  Based  Approaches  to  Security  and 
Defence  within  a  multinational  coalition. 
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Multinational  Experimentation  Strategy 


Nov  01 

Collaboration 

Planning 

Methodology 


Feb  03 
Knowledge 
Information 
Sharing 


Feb  04 
Effects 
Based 
Planning 


Feb/Mar  06 
Effects 
Based 
Operations 


Defence  R&D  Canada  •  R  &  D  pour  la  defense  Canada 


RD7  USJFCOM’s  Joint  Pathways 


Joint  Concept 

A  Joint  Concept  is  a  visualization  of  future  operations  that  describes  how  a  commander 
might  employ  capabilities  to  achieve  desired  effects  and  objectives. 


Joint  Concept  Development  Path 


Joint  Prototype  Path 


Joint  Prototype 

A  Joint  Prototype  is  a  model  suitable  for  evaluation  of  design,  performance,  or  production 
potential.  It  can  be  an  idea,  process,  organization,  planning  method,  product,  or  tool. 
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r^y  Potential  Way  Ahead  for  CFEC 


•  CFEC  needs  both  concept  development  and 
prototyping  venues  to  evolve  the  products 
associated  with  concept  development. 

•  Although  the  events  in  the  MNE  series  are  part  of 
J9’s  prototype  pathway,  they  are  generally  concept 
development  opportunities  for  the  other  nations. 


It  is  therefore  proposed  that  CWID,  an  environment 
traditionally  for  prototypes  about  to  be  taken  into 
operational  use,  could  also  be  suitable  for  prototype 
products  from  concept  development  environments. 
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Conclusions 
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The  case  studies  highlighted  two  technologies  that  have 
potential  for  concept  development  experiments. 

-  The  COP  21  Portal  provided  a  capability  for  net- 
centric  operations  for  an  operational  level 
headquarters.  CFEC  subsequently  considered  using 
the  COP  2 1  Portal  in  their  Knowledge  Management 
experimentation  for  MNE  4  and  has  since  acquired 
the  COP  2 1  Portal  for  further  experimentation. 

-  COPlanS  was  proposed  for  use  in  MNE  3  with 
consideration  for  employment  in  MNE  4.  The 
reconfigurable  features  of  this  application  allow  it  to 
be  applied  to  any  planning  process. 

Subsequent  to  a  successful  concept  development 
experiment,  tools  should  be  trialed  in  CWID  again  for 
validation  prior  to  integration  into  operational  systems. 
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Conclusions 
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•  CWID  provides  opportunities  for  exposing 
prototypes  to  the  operational  community. 

•  However,  there  are  limitations: 

-  No  single  event  can  meet  all  the  requirements  of 
the  CD&E  community. 


-  Likewise,  both  pathways  are  needed  to  produce 
well-rounded  concepts  and  tools. 

With  careful  planning,  venues  such  as  CWID  and 
MNE4  can  be  used  iteratively  to  progress  CD&E  in 
the  joint  environment. 
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